Electronic device and method for pushing information based on user emotion

ABSTRACT

A method for pushing information based on a user emotion including recordings of behavior habits of the user based on a number of predefined emotions within a predefined time period can be implemented in the disclosed electronic device. Based on each predefined emotion, a proportion of each behavior habit of the user is determined at the predetermined time intervals. The device determines information to be pushed according to a current user emotion and the proportions of the behavior habits of the user corresponding to the current user emotion, and the electronic device is controlled to push the determined information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No.201910810585.6 filed on Aug. 29, 2019, the contents of which areincorporated by reference herein.

FIELD

The subject matter herein generally relates to human-machineinteraction, and particularly to an electronic device and a method forpushing information based on a user emotion.

BACKGROUND

Users of smart electronic devices, such as smart phones, personalcomputers, etc., can receive a lot of information promoted or putforward by providers, such as news, game information, healthinformation, service information, etc. However, if the informationpushed by the providers does not match a user emotion, a user experiencewill be affected.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. The components in the drawings are notnecessarily drawn to scale, the emphasis instead being placed uponclearly illustrating the principles of the disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is a block diagram of an embodiment of an electronic device.

FIG. 2 is a block diagram of an embodiment of modules of an informationpushing system in the electronic device of FIG. 1.

FIG. 3 is a schematic view of an embodiment of a three-dimensional (3D)image of a user face captured by an image capturing device of theelectronic device of FIG. 1.

FIG. 4 illustrates a relationship table between relative distancesbetween muscles of a number of predefined parts and a number ofpredefined emotions.

FIG. 5 illustrates a flowchart of an embodiment of a method for pushinginformation based on a user emotion.

DETAILED DESCRIPTION

It will be appreciated that for simplicity and clarity of illustration,where appropriate, reference numerals have been repeated among thedifferent figures to indicate corresponding or analogous elements. Inaddition, numerous specific details are set forth in order to provide athorough understanding of the embodiments described herein. However, itwill be understood by those of ordinary skill in the art that theembodiments described herein can be practiced without these specificdetails. In other instances, methods, procedures, and components havenot been described in detail so as not to obscure the related relevantfeature being described. Also, the description is not to be consideredas limiting the scope of the embodiments described herein. The drawingsare not necessarily to scale and the proportions of certain parts havebeen exaggerated to better illustrate details and features of thepresent disclosure.

The present disclosure, including the accompanying drawings, isillustrated by way of examples and not by way of limitation. Severaldefinitions that apply throughout this disclosure will now be presented.It should be noted that references to “an” or “one” embodiment in thisdisclosure are not necessarily to the same embodiment, and suchreferences mean “at least one.”

Furthermore, the term “module”, as used herein, refers to logic embodiedin hardware or firmware, or to a collection of software instructions,written in a programming language, such as, Java, C, or assembly. One ormore software instructions in the modules can be embedded in firmware,such as in an EPROM. The modules described herein can be implemented aseither software and/or hardware modules and can be stored in any type ofnon-transitory computer-readable medium or other storage device. Somenon-limiting examples of non-transitory computer-readable media includeCDs, DVDs, BLU-RAY, flash memory, and hard disk drives. The term“comprising” means “including, but not necessarily limited to”; itspecifically indicates open-ended inclusion or membership in aso-described combination, group, series, and the like.

FIG. 1 illustrates an electronic device 1 in one embodiment. Theelectronic device 1 can be a smart phone, a smart watch, or a smartbracelet. The electronic device 1 can recognize user emotions, and pushinformation to the user based on his/her emotions.

The electronic device 1 includes, but is not limited to, a processor 10,a storage device 20, a display device 30, and an image capturing device40. FIG. 1 illustrates only one example of the electronic device 1.Other examples can include more or fewer components than illustrated orhave a different configuration of the various components in otherembodiments.

The processor 10 can be a central processing unit (CPU), amicroprocessor, or other data processor chip that performs functions inthe electronic device 1.

In at least one embodiment, the storage device 20 can include varioustypes of non-transitory computer-readable storage mediums. For example,the storage device 20 can be an internal storage system, such as a flashmemory, a random access memory (RAM) for temporary storage ofinformation, and/or a read-only memory (ROM) for permanent storage ofinformation. The storage device 20 can also be an external storagesystem, such as a hard disk, a storage card, or a data storage medium.

In at least one embodiment, the storage device 20 stores a number ofapplications installed in the electronic device 1.

In at least one embodiment, the display device 30 can be a touch screen.The display device 30 displays an operation interface of the electronicdevice 1.

In at least one embodiment, the image capturing device 40 can be a truedepth camera. The image capturing device 40 includes, but is not limitedto, a floodlight sensing member 401, an infrared lens 402, and a dotprojector 403.

When the image capturing device 40 detects that at least one object isapproaching, the floodlight sensing member 401 projects infrared lighttoward the at least one object. The infrared lens 402 receives infraredlight reflected by the at least one object, forms an infrared image ofthe at least one object, and determines whether the at least one objectincludes a human face. When the at least one object includes a humanface, the dot projector 403 projects a number of light dots towards thehuman face. The infrared lens 402 receives the light dots reflected bythe human face and generates a 3D (three-dimensional) image of the humanface.

As illustrated in FIG. 2, the electronic device 1 runs an informationpushing system 100. The information pushing system 100 at least includesan image capturing module 101, a recognizing module 102, an establishingmodule 103, a recording module 104, a determining module 105, adetecting module 106, and a pushing module 107. The modules 101-107 canbe collections of software instructions stored in the storage device 20of the electronic device 1 and executed by the processor 10. The modules101-107 also can include functionality represented as hardware orintegrated circuits, or as software and hardware combinations, such as aspecial-purpose processor or a general-purpose processor withspecial-purpose firmware.

In at least one embodiment, the electronic device 1 can pre-determine arelationship between relative positions of facial muscles of the userand user emotions, according to machine learning and big data analysis.

The image capturing module 101 is used to control the image capturingdevice 40 to capture a number of 3D images of a user face based on anumber of predefined emotions.

In at least one embodiment, when the user is using the electronic device1, the image capturing module 101 controls the image capturing device 40to capture the number of 3D images of the user in response to useroperations. The image capturing device 40 captures a 3D image based oneach of the number of predefined emotions.

In at least one embodiment, the user operations at least include facingthe display device 30 and performing at least one expressioncorresponding to each predefined user emotion. In at least oneembodiment, the predefined user emotions at least include happiness,sadness, anger, and anxiety.

When the user faces the display device 30 and makes a facial expressioncorresponding to a predefined emotion, the image capturing module 101controls the floodlight sensing member 401 to transmit infrared light toat least one object, and controls the infrared lens 402 to receive theinfrared light reflected by the at least one object, and to form aninfrared image. The image capturing device 40 is controlled to detectwhether the at least one object includes a user face. When the at leastone object includes the user face, the image capturing module 101further controls the dot projector 403 to project a number of light dotsto the user face, and controls the infrared lens 402 to receive lightspots reflected by the user face, and form an image in a point cloudform including depth data of different positions of the user face. A 3Dimage of the user based on one of the number of predefined emotions isgenerated according to the image in the point cloud form.

In detail, the infrared lens 402 determines a number of distancesbetween various parts (e.g. forehead, eyes, cheek, mouth, and chin) ofthe user face and the image capturing device 40, according to a timeinterval between transmission of light dots and receiving reflectedlight dots, and a propagation speed of the light dot. The infrared lens402 further determines a 3D structure of the user face according to thedetermined distances between the various parts of the user face and theimage capturing device 40. The 3D structure is equal to the depth imageof the user face. The infrared lens 402 further generates the 3D imageof the user face according to the 3D structure.

In at least one embodiment, the image capturing device 40 can capturethe number of 3D images of the user face based on the number ofpredefined emotions (i.e. happiness, sadness, anger, and anxiety)through above methods.

The recognizing module 102 is used to recognize relative distancesbetween muscles of a number of predefined parts according to each 3Dimage of the user face captured by the image capturing device 40.

In at least one embodiment, the number of predefined parts at leastinclude a forehead, an eye, a mouth, and a cheekbone.

Referring to FIG. 3, the recognizing module 102 determines the foreheadA, the eye B, the cheekbone C, and the mouth D in the 3D image of theuser face by feature recognition, and calculates a distance AB betweenthe forehead and the eye, a distance BC between the eye and thecheekbone, and a distance CD between the cheekbone and the mouth. Thedistances AB, BC, and CD are actual distances in the 3D image of theuser face between the number of predefined parts.

In at least one embodiment, the relative distances between the musclesof the number of predefined parts represent the relative positionsbetween the muscles of the predefined parts and reflect the useremotions.

The establishing module 103 is used to establish a relationship tablebetween the relative distances between the muscles of the predefinedparts and the number of predefined emotions.

Referring to FIG. 4, in at least one embodiment, the establishing module103 establishes the relationship table by associating each combinationof the relative distances between the muscles of the number ofpredefined parts with a corresponding user emotion, and stores therelationship table to the storage device 20.

For example, when the user emotion is the happiness, the combination ofthe relative distances includes the distance AB, the distance BC, andthe distance CD, the distance AB can be 2.5 cm, the distance BC can be 1cm, and the distance CD can be 1.8 cm.

In other embodiments, the establishing module 103 can control the imagecapturing device 40 to capture a number of 3D images of the user facebased on each predefined emotion. The recognizing module 102 recognizesa number of combinations of relative distances between the muscles ofthe number of predefined parts, according to the captured 3D images. Therecognizing module 102 further determines a number of ranges of therelative distances based on each predefined emotion according to thenumber of combinations of relative distances. In at least oneembodiment, each range of relative distances is between a minimum valueand a maximum value of the relative distances. The establishing module103 further establishes the relationship table according to the numberof ranges of the relative distances.

For example, when the user emotion is the happiness, the number ofranges of the relative distances includes a distance range AB, adistance range BC, and a distance range CD, the distance range AB can be2.45 cm-2.55 cm] the distance range BC can be 0.95 cm-1.05 cm, and thedistance range CD can be 1.75 cm-1.85 cm.

The recording module 104 records behavior habits of the user based onthe number of predefined emotions within a predefined time period.

In at least one embodiment, the behavior habits of the user at leastinclude operations performed on the electronic device 1 and actionsperformed by the user. The predefined time period can be one week. Theoperations at least include activating an application or using anapplication.

In at least one embodiment, the image capturing device 40 can detectuser actions, such as smoking a cigarette. In detail, the imagecapturing device 40 can detect the user actions by capturing imagesand/or videos of the user, and search Internet data using the capturedimages and/or videos according to big data analysis, thus acquiring atype of the user action, such as the smoking action.

In at least one embodiment, the recording module 104 counts theoperations of activating/using the application and the actions performedby the user based on the number of predefined emotions within one week,the behavior habits of the user based on the number of predefinedemotions within one week are thus recorded.

The determining module 105 is used to determine a proportion of eachbehavior habit of the user based on each predefined emotion according tothe recorded behavior habits.

For example, when the user emotion is the happiness, a percentage oftimes of activating/using WeChat is 70%, and a percentage of times ofactivating/using Weibo is 25%. When the user emotion is the sadness, apercentage of times of activating/using a music application is 65%, anda percentage of times of activating/using a game application is 30%.When the user emotion is the anger, a percentage of times of smoking is60%, and the percentage of the times of activating/using the musicapplication is 35%. When the user emotion is the anxiety, the percentageof the times of activating/using the music application is 50%, and thepercentage of the times of activating/using the game application is 20%.

The detecting module 106 is used to detect a current user emotion atpredetermined time intervals.

In at least one embodiment, the predetermined time interval can bethirty minutes. The detecting module 106 detects the current useremotion, by controlling the image capturing device 40 to capture the 3Dimage of the user face. The detecting module 106 further recognizes therelative distances between the muscles of the number of predefined partsin the 3D image. The detecting module 106 further determines the currentuser emotion according to the relative distances and the relationshiptable.

In detail, the relative distances between the muscles of the number ofpredefined parts in the 3D image forms a combination of distances. Thedetecting module 106 compares the formed combination of distances withthe number of combinations of distances in the relationship table asillustrated in FIG. 4. when the formed combination of distance is equalto one of the number of combinations of distances, or a differencebetween the formed combination of distances and one of the number ofcombinations of distances is within a predefined range, the detectingmodule 106 determines that a predefined emotion corresponding to thecombination of distances is the current user emotion, according to therelationship table.

Furthermore, in other embodiments, the electronic device 1 furtherincludes a heart rate sensor (not shown), and the storage device 20further pre-stores a heart rate range corresponding to each of thenumber of predefined emotions. The detecting module 106 further controlsthe heart rate sensor to sense a heart rate of the user at thepredetermined time intervals, and determines whether the sensed heartrate is within the heart rate range corresponding to the current useremotion. When determining that the sensed heart rate is within the heartrate range corresponding to the current user emotion, the detectingmodule 106 confirms that the predefined emotion is the current useremotion.

The determining module 105 is further used to determine information tobe pushed by the electronic device 1, according to the current useremotion and the proportion of the behavior habits of the usercorresponding to the current user emotion.

In at least one embodiment, the information to be pushed by theelectronic device 1 can be determined according to the behavior habitwith the highest proportion of the current user emotion. In detail, whenthe behavior habit with the highest proportion of the current useremotion is activating/using an application, the determining module 105determines the application as the information to be pushed, andactivates the application. For example, if the detecting module 106detects that the current user emotion is the happiness, the determiningmodule 105 determines that WeChat is the information to be pushed, andactivates WeChat.

When the behavior habit with the highest proportion of the current useremotion is an action performed by the user, the determining module 105determines that a voice information for prompting the user to performthe action should be the information to be pushed.

In other embodiments, the determining module 105 can further determinewhether the action to be performed is beneficial to the user. When theaction is beneficial to the user, the determining module 105 determinesthe voice information for prompting the user to perform the action isthe information to be pushed.

When the action is not beneficial to the user, the determining module105 further determines whether the behavior habit with the secondhighest proportion of the current user emotion is activating/using acertain application or an action performed by the user. When thedetermined behavior habit is activating/using such application, thedetermining module 150 determines that the application and the voicemessage for prompting the user not to perform the action are theinformation to be pushed.

For example, if the user current emotion is the anger, the determiningmodule 105 determines that the music application and the voice prompt“Smoking is harmful, you should listen to music” as the information tobe pushed. When the determined behavior habit is the action performed bythe user, the above process is repeated.

The pushing module 107 is used to control the electronic device 1 topush the information determined by the determining module 105.

For example, when the determined information is the WeChat application,the pushing module 107 pushes the WeChat application, that is, activatesthe WeChat application. When the determined information is the musicapplication and the voice prompt “Smoking is harmful, you should listento music”, the pushing module 107 pushes the music application, that is,activates the music application, and controls the electronic device 1 tooutput such voice prompt.

In other embodiments, the detecting module 106 is further used todetermine a user emotion when the user is accessing an application. Ifthe user emotion when accessing the application is detected to be thehappiness, the pushing module 107 pushes the information with similarcontent as the accessed content. For example, when the user is browsingmerchandise via a shopping application, and the detecting module 106detects that the user emotion is the happiness, the pushing module 107pushes similar merchandises to the user, thereby providing more choicesto the user.

In other embodiments, the image capturing device 40 can be exposed to anexternal environment. When a face of a person around the user enters acapturing range of the image capturing device 40, the detecting module106 detects the emotion of the person. If the emotion (e.g. anger andanxiety) of the person has a potential threat to the user, the pushingmodule 107 pushes a prompt message to prompt the user to pay attention.The prompt information can be popup information or vibration.

FIG. 5 illustrates a flowchart of an embodiment of a method for pushinginformation based on a user emotion. The method is provided by way ofexample, as there are a variety of ways to carry out the method. Themethod described below can be carried out using the configurationsillustrated in FIGS. 1-2, for example, and various elements of thesefigures are referenced in explaining the example method. Each blockshown in FIG. 5 represents one or more processes, methods, orsubroutines carried out in the example method. Furthermore, theillustrated order of blocks is by example only and the order of theblocks can be changed. Additional blocks may be added or fewer blocksmay be utilized, without departing from this disclosure. The examplemethod can begin at block 501.

At block 501, the image capturing module 101 controls the imagecapturing device 40 to capture a number of 3D images of a user facebased on a number of predefined emotions.

At block 502, the recognizing module 102 recognizes relative distancesbetween muscles of a number of predefined parts according to each 3Dimage of the user face captured by the image capturing device 40.

At block 503, the establishing module 103 establishes a relationshiptable between the relative distances between the muscles of thepredefined parts and the number of predefined emotions.

At block 504, the recording module 104 records behavior habits of theuser based on the number of predefined emotions within a predefined timeperiod.

At block 505, the determining module 105 determines a proportion of eachbehavior habit of the user based on each predefined emotion according tothe recorded behavior habits.

At block 506, the detecting module 106 detects a current user emotion atpredetermined time intervals.

At block 507, the determining module 105 further determines informationto be pushed by the electronic device 1 according to the current useremotion and the proportion of the behavior habits of the usercorresponding to the current user emotion.

At block 508, the pushing module 107 controls the electronic device 1 topush the information determined by the determining module 105.

It is believed that the present embodiments and their advantages will beunderstood from the foregoing description, and it will be apparent thatvarious changes may be made thereto without departing from the spiritand scope of the disclosure or sacrificing all of its materialadvantages, the examples hereinbefore described merely being embodimentsof the present disclosure.

What is claimed is:
 1. An electronic device comprising: at least oneprocessor; an image capturing device coupled to the at least oneprocessor; and a storage device coupled to the at least one processorand storing instructions for execution by the at least one processor tocause the at least one processor to: record behavior habits of a userbased on a plurality of predefined emotions within a predefined timeperiod; determine a proportion of each of the behavior habits of a userbased on each of the plurality of predefined emotions according to therecorded behavior habits; detect a current user emotion at predeterminedtime intervals; determine information to be pushed by the electronicdevice according to the current user emotion and the proportion of eachof the behavior habits of the user corresponding to the current useremotion; and control the electronic device to push the determinedinformation.
 2. The electronic device according to claim 1, wherein theat least one processor is further caused to: control the image capturingdevice to capture a plurality of 3D images of a user face based on theplurality of predefined emotions; recognize relative distances betweenmuscles of a plurality of predefined parts according to each of theplurality of 3D images of the user face captured by the image capturingdevice; and establish a relationship table between the relativedistances between the muscles of the plurality of predefined parts andthe plurality of predefined emotions.
 3. The electronic device accordingto claim 2, wherein the at least one processor is further caused to:control the image capturing device to capture the plurality of 3D imagesof a user face; recognize the relative distances between the muscles ofthe plurality of predefined parts in the 3D image; and determine thecurrent user emotion according to the relative distances and therelationship table.
 4. The electronic device according to claim 3,wherein the plurality of predefined parts comprise a forehead, an eye, amouth, and a cheekbone; wherein the at least one processor is furthercaused to: determine the forehead, the eye, the cheekbone, and the mouthin the 3D image of the user face by feature recognition; and calculate adistance between the forehead and the eye, a distance between the eyeand the cheekbone, and a distance between the cheekbone and the mouth.5. The electronic device according to claim 2, wherein the at least oneprocessor is further caused to: associate each combination of therelative distances between muscles of the plurality of predefined partswith a corresponding user emotion; and store the relationship table tothe storage device.
 6. The electronic device according to claim 1,wherein the image capturing device comprises a floodlight sensingmember, an infrared lens, and a dot projector.
 7. The electronic deviceaccording to claim 6, wherein the at least one processor is furthercaused to: control the floodlight sensing member to transmit infraredlights to at least one object; control the infrared lens to receiveinfrared lights reflected by the at least one object and to form aninfrared image; control the image capturing device to detect whether theat least one object includes a user face; wherein when the at least oneobject includes the user face, the at least one processor is furthercaused to: control the dot projector to project a plurality of lightdots towards the user face; control the infrared lens to receive thelight spots reflected by the user face and form an image in a pointcloud form including depth data of different positions of the user face;and generate a 3D image of the user face based on one of the pluralityof predefined emotions according to the image in the point cloud form.8. A method for pushing information based on a user emotion implementedin an electronic device comprising: recording behavior habits of a userbased on a plurality of predefined emotions within a predefined timeperiod; determining a proportion of each of the behavior habits of auser based on each of the plurality of predefined emotions according tothe recorded behavior habits; detecting a current user emotion atpredetermined time intervals; determining information to be pushed bythe electronic device according to the current user emotion and theproportion of each of the behavior habits of the user corresponding tothe current user emotion; and controlling the electronic device to pushthe determined information.
 9. The method according to claim 8, furthercomprising: controlling an image capturing device to capture a pluralityof 3D images of a user face based on the plurality of predefinedemotions; recognizing relative distances between muscles of a pluralityof predefined parts according to each of the plurality of 3D images ofthe user face captured by the image capturing device; and establishing arelationship table between the relative distances between the muscles ofthe plurality of predefined parts and the plurality of predefinedemotions.
 10. The method according to claim 9, further comprising:controlling the image capturing device to capture the plurality of 3Dimages of a user face; recognizing the relative distances between themuscles of the plurality of predefined parts in the 3D image; anddetermining the current user emotion according to the relative distancesand the relationship table.
 11. The method according to claim 10,wherein the plurality of predefined parts comprise a forehead, an eye, amouth, and a cheekbone, and the method of recognizing the relativedistances between the muscles of the plurality of predefined parts inthe 3D image comprises: determining the forehead, the eye, thecheekbone, and the mouth in the 3D image of the user face by featurerecognition; and calculating a distance between the forehead and theeye, a distance between the eye and the cheekbone, and a distancebetween the cheekbone and the mouth.
 12. The method according to claim9, wherein the method of capturing a plurality of 3D images of a userface based on the plurality of predefined emotions comprises:controlling a flood light sensing member to transmit infrared lights toat least one object; controlling an infrared lens to receive infraredlights reflected by the at least one object and to form an infraredimage; controlling the image capturing device to detect whether the atleast one object includes a user face; wherein when the at least oneobject comprises a user face, the method of capturing a plurality of 3Dimages of a user face based on the plurality of predefined emotionsfurther comprises: controlling a dot projector to project a plurality oflight dots towards the user face; controlling the infrared lens toreceive the light spots reflected by the user face and form an image inpoint cloud form including depth data of different positions of the userface; and generating a 3D image of the user face according to the imagein the point cloud form.
 13. The method according to claim 9, whereinthe method of establishing a relationship table between the relativedistances between the muscles of the plurality of predefined parts andthe plurality of predefined emotions comprises: associating eachcombination of the relative distances between muscles of the pluralityof predefined parts with a corresponding user emotion; and storing therelationship table to the storage device.